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ABSTRACT 

This discussion of the quality and impact of the 
prior knowledge state presents theoretical and empirical 
explorations. The theoretical discussion offers a distinct approach 
to the analysis of prior knowledge based on an extensive literature 
review of theory, models, and practice about the structure of 
knowledge. This base is used to define a set of dimensions that are 
useful in constructing knowledge profiles. The dimensions are: (1) 
cognitive-psychological ; (2) educational -psycho logical ; (3) 
psychometric; and (4) content-based. In a discussion of empirical 
results, the dimensions are used to analyze the knowledge profiles of 
591 economics students at the Open University of the Netherlands and 
the Maastricht University with low and high levels of prior knowledge 
of economics. In this samplt), profile analysis did not help to reveal 
specific significant differences between the two subpopulat ions , 
although the research was helpful in defining and opera t i onal izing a 
new approach to the analysis of prior knowledge states. Ten tables 
and 23 figures illustrate the investigation. (Contains 31 
references.) (SLD) 
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OTIC RESEARCH REPORFS. 

The Open University is responsible .or developing and offering open, higher distance education in which special 
attention is paid to innovations in educational technology. The research in this field is concentrated in "OTIC", that is 
the Centre for Educational Technological Innovations (Onderwijs Technologisch Innovatie Centrum). 
OTIC is also engaged in running projects for other institutions. Here the Centre makes use of OTIC*s knowledge and 
experience acquired in research and development. 

The series of OTIC Research Reports consists of publications of the OTIC research projects and aims mainly at an 
audience of fellow researchers. 



THE RESEARCH PROJECT ON EVALUATION AND TEST-FUNCTIONALITIES 

The project "Evaluation and Test-^nctionalities" focuses on the problems caused by the wide diversity of students and 
the problems with individual and flexible learning-processes of these students. The project leads to an integration of: 

- the results of the project "Prior Knowledge"; 

- the developments of the "Computer Assisted Testing"-project; 

- the developments of "Adaptive Testing" and the IRT-applications Otcm-Responds Theory); and 

- the experience of the Open University with the development and use of TSS (Test service systems). 

The main objectives are: (1) to get a discernment of the test and evaluation problems in the open-learning system; (2) 
the generation of the guide-lines, specifications, and technological instruments concerned with the use of prior 
knowledge and experience, flexible testing and the supervision on students during the learning process, and (3) the 
development of instruments wdiich can be usefiil in solving the given teaching problems. 
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1 Introduction 

The analysis of die quality and impact of the prior knowledge state has been the major focus in a large part 
of our earlier research. In analyzing the prior knowledge state, we did especially focus on the structure of 
the prior knowledge state along a cont^t dim^ion. 

In the theoretical part of this text, we discuss - in short - a distinct q>proach towards the analysis of the 
prior knowledge state^ This approach is based on an extensive analysis of the literature in relation to 
theories, models and practice-based strategies about the "structure of knowledge". This base is exploited to 
define a set of "dimensions" that are helpfid to construct "knowledge profiles". Four types of dimensions 
are illustrated : cognitive psychological dim^ions, educational-psychological dimensions, psychometrical 
dim^isions and cont^t-based dimensions. 

In the enq)irical part of this text, the dimensions are used to analyze the knowledge profiles of students with 
low and high levels of the prior knowledge state. The results of this analysis might not only be helpiul to 
detect specific differences in the mastery of components of the prior knowledge state between both student 
populations, but might also be helpfiil to provide further evidence about the validity of the theoretical 
knowledge profile dimensions. 

2 Theoretical background 

2, 1 The Structure of Knowledge 

From an instructional-psychological point of view, the structure-of-knowledge problem should be 
investigated in order to find out more efficient ways for using instructional technology. Our search for 
means to handle the prior knowledge state showed that one should take account differ^t components of the 
prior knowledge state. The concept of "components" refers towards a structure in the knowledge base of 
the learner. 

Our earlier research was helpfiil to detect such compon^ts of the prior knowledge state along the content 
dimension. But it was also suggested that the differentiation of components of the prior knowledge state 
along other dimensions is needed to be helpful to interfere and diagnose educational practice (Dochy and 
Valcke, 1991; Dochy and Valcke, 1991b; Wagemans, Valcke and Dochy, 1991). 

The issue of the "structure of knowledge" has be^ debated from a variety of theoretical points of view : 
cognitive psychology, epistemology, philosophy, etc. At the more pragmatic level, the issue has also been 
of prime inqiortance in applied sciences like instructional psychology, curriculum development theories and 
psychometry. 

Disciplines like cognitive psychology, educational psychology, artificial intelligence, etc. - have - from their 
points of view - highlighted the "structure of knowledge" resulting in a puzzling variety of approaches, 
focuses, models, theories, research attenq)ts, .... A rq)resentative sanq)le of authors comprises e.g. 
Avsubel (1968), de Groot (1946), Mayer (1979), Reigeluth and Stein (1983). 

It should be noted that our primary focus in using these theories originates from an information processing 
view on learning (Sternberg, 1985a & 1985b). The main reason for this is that we stress a dynamic 
approach towards the structure (knowledge acquisition) of knowledge, which is in particular advocated in 
this view. If we summarize the variety of approaches, four main types of dimensions to structure 
knowledge can be conceptualized : 



* A more elaborated version of the theoretical ba&e of the knowledge profile dimensions can be found in : Dochy & Valcke (1991a). 
Validation of Knowledge Profile Dimensions : Looking for empirical Evidence. OTIC Research report 33. Heerlen : Ou-OTIC. 
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Content related dimensions 
Educational dimensions 
Epistemological dimensions 
Item characteristic dimensions 



As discussed in another publication (Dochy and Valcke> 1991)> some of these dimension are of a 
hierarchical nature. 

2,2 Knowledge Profiles 

As such, the concept of 'knowledge profiles* is not found in literature. Only 'student profiles' (Wolf, et, 
al., 1991) and 'cognitive profile' (Letteri, 1980) have some similarity in meaning. This is certainly the case 
for the studies by Letteri et. al. (1980, 1982). The conc^t 'profile' is derived firom the practice, coinmon 
in educational research, of plotting as a graph or profile the scores of a person as raw scores or as 
standardized scores (Keeves, 1988). In analyzing research findings, comparisons are made between persons 
or groups in terms of a set of measurements on specific related aspects. For each person or group a profile 
is obtained on a set of parameters. The comparison between profiles of persons is known by the generic 
term 'profile analysis' . 

Figure 1 shows the relationship between some key concq>ts. A "dimension" is used to construct a 
knowledge profile. Each dim^ion represents an approach towards the structure of knowledge. The 
structure con^n^ts are named "parameters". 




Figure 1: Example of a profile 

From an instructional psychological point of view, knowledge profiles can give practical indications of 
student achievemwit and learning in order to direct the learning process. In a recent overview of stud^t 
assessment. Wolf et. al. (1991) advocate this t^proach. According to these authors, there is a need for new 
educational psychometrics capable of answering the much changed question of educational achievement. 
These changes are the new premises, the multiple paths towards the prior knowledge state, more 
developmental oriented assessments and the ascertainment that students enter school with widely varying 
backgrounds. In our terms, we take account of these changes by trying to identify multiple components of 
the prior knowledge state, by implementing prior knowledge state tests and by intending to use these tests as 
progress tests administered several times a year. In this context it is necessary to come to an agreement on 
the relevant parameters to describe student performance and it is critical to develop ways of looking at 
'student profiles': " imless we develop these kinds of differentiated portraits of student performance within a 
domain, it is difficult to envision student assessment ever informing, rather than merely measuring, the 
educational process" (Wolf, et. al., 1991). 
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2.3 Overview of Knowledge Profile Dim^ions 

Only those dim^ions/parameters are reviewed that have be^ retained after their discussion and analysis in 
our earlier publication "Validation of Knowledge Profile Dimensions : Looking for empirical Evidence". 
If dim^ions are based on a model or theory » only short details will be reported. 

The first dimensions are classified according to common models of economics. Other dimensions are based 
on theories of knowledge repres^tation» knowledge structure, learning theories, text repres^tation models 
and psychometric theory. 



2.3.1 Contrat Dim^isions 
Economics subdonmins dimensicq 



''Content" is a commonly used dimension to categorize domain knowledge. Classification based on the 
parameter 'subdomains' refers to the subdivision of the economics-domain into "subject matter blocks'* that 
are standard within the science of economics. Our dimension structure is e.g. based on the curriculum 
structure of the University of Maastricht and reflects 9 parameters : 



1. R^oiting 

2. Innancing 

3. Otganiz&tion 

4. Marketing 

5. Macro-economicB 

6. MicrcHeconomics 

7. Public finances 

8. Internationa] economic afTairs 

9. Bebaviourial and social sciences 



Curriculum level dimension 

Some parts of the content of a science are supposed to be mastered by the students at certain moments 
during their study. These moments are called the curriculum levels (first and second year). These levels 
are subsequent^ but too broad to be supposed hierarchical. 



1 . First year level 

2. Second year iovel 



Curriculum accent dimension 



Within economics it is common to differentiate between two main streams, representing a different accent, 
i.e. general economics and business administration on the one hand and quantitative economics on the other 
hand. 



1 . General econonucs and business admimstration 

2. Quantitative econonucs 
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2.3.2 Cognitive Psychological Dim^ions 
Node relation dimension 

Knowledge r^resentation, as used in schema theories (Dochy and Bouwens» 1990)» takes certain 
propositions or nodes as a starting point. A proposition is the smallest unit that can be qualified as true or 
false in a statem^t. AccordiQg to rnost schema theories there are five kinds of nodes : Physical State (PS» 
statem^t that refers to an ongoiag state in the physical or social world) » Physical Event (PE» statement that 
refers to a state change in the physical or social world). Internal State (IS» statement that refers to an 
ongoing state of knowledge, attitude, or belief in a character). Internal Ev^t (IE, refers to a state change ia 
knowledge, attitude or belief in a character). Goal (G, statem^t that refers to an achieved or unachiev^ 
state that a person wants) and Style (S, statem^t that refers to details about the style or manner in which an 
action or event occunied. 



l.O- G REASON 



2. PS-O INrriATE 
IS-O 
PE-O 



3. PS - PE CONSEQOTNCE 
IS-PE 
PE-PE 
m- PE 
O - PE 
PS-PS 
IS-PS 
PE-PS 

m-ps 

G -PS 



4. PE - S/O MANNER 

m-s/o 
OE - s/o 



5. PS -PS PROPERTY 



The "Node Relation" dimension is based on characteristics of the interrelations between propositions, called 
node relation or arc parameters: Reason (R, a Goal node is a reason for another Goal node). Initiate (I, a 
State or Event initiates another Goal node). Consequence (C, a State, Event or Goal node that has the 
consequence of another State or Event node). Manner (M, an Event or Goal node occurs with some style). 
Property (P, a person, object or ^tity has some property that is a State node) (see also Dochy and 
Bouw^, 1990). These arc parameters are not of a hierarchical nature. 

2.3.3 Educational-Psychological Dimensions 

The theoretical base of these two dimensions - i.e. behaviourial and cont^t dim^sion - is found in 
Component Display Theory (CDT, Merrill, 1983), Taxonomy theories (De Block, 1986 and Bloom, 1956) 
and GagnS's theoretical classification (1985). 

Behaviourial dimension 

The known distinction between declarative and procedural knowledge is further operationalised at this stage 
into the parameters ^to know, to understand, and to apply \ These parameters are also perceived as 
equivalent to the concepts 'recognition, reproduction and production". Items can be classified as measuring 
the appreciation, the recognition and the reproduction of information (declarative) or measuring production 
or applications (interpretative, convergent, divergent or evaluative productions^ procedural) (Keeves, 1988). 
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The three parameters do also correspond with taxonomic levels proposed by several educationalists as 
Bloom, Guilford, De Corte and De Block (cf. Keeves, 1988). Most researchers agree that these paramefters 
are hierarchical in nature. 



1. Know 


1. Declarative 


2. Understand 




3. Apply 


2. Procedural 



Content dimension 

Along the content dimension we differentiate five parameters : fects, concq)ts, relations, structures aud 
methods. This is in accordance with e.g. the work of Guilford v/hea he .refers to product parameters 
(Keeves, 1988). These parameters are widely accepted as being hierarchical (Keeves, 1988). 



1. Facts 

2. Concepts 

3. Relations 

4. Stmct'fres 

5. Methods 



E pistemological dimension 

Based on the levels of knowledge representation of Brachman and Schmolze (1985), five parameters can be 
differentiated along a typical dimension. These parameters can also be considered as the most expropriate 
combinations of behaviourial- and content levels, as clarified between brackets : knowledge id^tification 
(identifying facts and concepts), knowledge concq)tualisation (insight in concepts), ^istemol jical analysis 
(to know and understand, relations and structures), logical analysis (to know and understand methods), 
in^lementational analysis (application of methods). These levels are considered as hierarchical since they 
are a combination of the hierarchical b^viotirial and content level. 



1. Knowledge identi^cation 

2. Knowledge conceptualisation 

3. ^istemological analysis 

4. Logical analysis 

5. Inq>Iementational analysis 



2.3.4 Item Characteristics Dimensions 
Number of propositions dimension 

A proposition is the smallest unit that can stand as a separate assertion which can be judged as true or false. 
In schema theories (Dochy and Bouw^, 1990), propositions or nodes have a core function in the structure 
of schemata. It is assumed that the amount of propositions determines the degree of structure needed to 
answer the item correctly. Three parameters have been identified in relation to this dimension : 



1 . < 5 propoutions 

2. > 4 < 10 propositions 

3. > 9 propositions 
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Information level dim^ion 

The "stem" of an item is the general information which is givra and which must not be evaluated. This 
correct information precedes the questions for which this information should be taken into account. A stem 
can be connected to one or more subsequent questions. Therefore* the spatial and logical dist:;nce between 
the general information part of an item and the question part is larger than for simple items without a stem. 



1. Itenu with a etem 

2. Itenu without a stem 



Representation level dimension 

Following the classification used in the research of Boekaerts (1979)* i.e. visual* verbal and symbolic 
r^res^tation* we distinguish four parameters along this dimension. These parameters are also closely 
related to the four content levels of Guilford^s structure of intellect model: ^gural* symbolic* semantic (the 
verbal fector) and behaviourial (nonverbal information) and the Twyman (1985) categories : verbal* pictorial 
and schematic. 

Test-items are always based on textual information rq)resentation* but can be enhanced* enriched or 
documented with information of an other representation category : 



1. Textual-graphical 

2. Textual 

3. Schematic 

4. Textual-symbotic 



2.4 LE and HE : What are the differences in their prior knowledge state ? 

Knowledge state profiles can be helpful to differentiae betwe^ students with low levels of expertise (LE) 
and students with high levels of expertise (HE). This might be especially interesting since this analysis can 
help us to detect what specific aspect of the structure of knowledge is especially differing between students 
with low (LE) or high (HE) expertise. The latter can be derived from the results of profile analysis when 
the outcome clearly shows which dimensions and/or which parameters along the dimensions can 
describe/explain the differences in expertise. 

Of course* the reader can comment on our approach by stating that it is obvious there will be overall 
significant differences between the mean scores of student with low and high expertise. We agree with this 
comment but draw the reader^s attention to the possibility that - since we work with multiple parameters 
along multiple dimensions - we can expect certain particularities in the pattern of differences (the profiles) 
for each specific dimension. So* we are not* in the first instance* interested in the sigi^Qcance* of 
differences between the results of both populations* but we are rather interested in the ^pattern* in the 
significant differences : Where are these differences more important* more explicit ? 
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3 



Research design 



3.1 Hypotheses 

Taking into account the theoretical base of the present study, the following main hypothesis can be stated : 

'^Students with low and high levels of the prior knowledge state are different in terms of the variables 

along a variety of knowledge profiles. " 

Since up to 10 profile dimensions will be used, the main hypothesis can be split up into a set of 10 
subhypotheses : 

Students with low and high levels of the prior knowledge state are different in terms of the 
subtopics knowledge profile. 

Students with low and high levels of the prior knowledge state are different in terms of the course 
levei knowledge profile. 

Students with low and high levels of the prior knowledge state are different in terms of the 
curriculism accent knowledge profile. 

Students with low and high levels of the prior knowledge state are different in terms of the node 
relation knowledge profile* 

Students with low and high levels of the prior knowledge state are different in terms of the 
behaviourial level knowledge profile. 

Students with low and high levels of the prior knowledge state are different in terms of the content 
le?d knowledge profile. 

Students with low and high levels of the prior knowledge state are different in terms of the 
epistesnologicaS levd knowledge profile. 

Stud^ts with low and high levels of the prior knowledge state are different in terms of the 
representation level knowledge profile. 

Students with low and high levels of the prior knowledge state are different in terms of the number 
of propositions knowledge profile. 

Students with low and high levels of the prior knowledge state are different in terms of the stem 
knowledge profile. 

3.2 Research Instruments 

As will be explained below, a domain specific knowledge state test was administered to the research 
population. This test consists of 154 items. The test covers the whole domain of economics to be studied 
at university level. This test consists of multiple-choice questions which can be answered with true/false or 
?. The ?-altemative is taken as a third alternative in order to prevent guessing. 

Characteristics of the test suggest that the determination of certain psychometric qualities might be a 
problem. There is no problem in relation to validity since the test clearly represents - to a very large extent 
- the "economics" domain and has been developed by a team of domain experts. 

On the other hand, determining the reliability of the test induces some specific problems. If we calculate 
the alpha-coefficient, the test can be considered as very reliable : a = .9302. 
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Table 1 : 

a-coefficients for the course subtopics and curriculum accent dimensions 
and mean a-coefficients 



PARAMETERS 


or 






Repoiting 


.5739 


18 




Financing 


.6449 


18 




Ofganizatton 


.6922 


18 




Marketing 


.6292 


18 




Macfo-eccnomics 


.7069 


25 


.631 


Microeconomics 


.7420 


25 




Public finances 


.5101 


11 




Intern, economic affairs 


.5543 


11 




Behaviourial & social sciences 


.6287 


10 




General economics & B.A. 


.9270 


139 




Quantitative economics 


.4467 


15 


.686 



But this high reliability level is marred by the fact that the test is very long (154 items); thus resulting rather 
easily in a high a-coefficient« More important, calculation of the ot-coefficient supposes the test to be 
homogeneous. Mostly tests are homogeneous at the content level. The delineation of the knowledge profile 
dimensions above, indicates that this basic assumption to calculate the a-coefficient has been violated. 
A solution to this problem might be to check the reliability of subparts of the test, making use of the 
knowledge profile dimensions. 

Calculation of a was repeated for two of these dimensions (subdomains dim^ion and curriculum accent 
dimension), ic order to be able to calculate a mean reliability score (mj. 

When reorganising the test into more homogeneous subparts, a-coefficient and the mean a*coefficient was 
calculated. The results of this procedure are scunmarized in table 1. To be able to judge the figures in a 
better perspective, the number of items each subgroup of items consists of is also given. 
Mean a seems to be > .63. This reliability score is - taking into account the restricted number of items in 
certain subparts of the tesi - acctptable for our research purposes. 

3.3 Research Population and Procedure 

3.3.1 Research Population 

The test was administered to a sample of economics students studying at the Dutch Open university and the 
Maastricht University. This sample consisted of 591 stud^ts. 

The mean score for the test was used in order to derive the students with the 50% Highest scores (HE) and 
the stud^ts with the 50% Lowest scores (LE) (N-290 and N- 301). A further distinction has been made 
in order to derive the 25% students with the highest scores within the High score group (N= 144) and the 
25% students with the lowest scores within the Low score group (N- 144). 
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3.3.2 Research Procedure 

The domain spscific knowledge state test was administered to the sample of Ou-stud^ts. The raw scores 
for the test items were recoded in order to gather a maximum of information in relation to mastery or mm^ 
mastery of the domain specific knowledge. After recoding^ a general economics-score for the entire test 
was calculated. 

In a next st^, all items were classified along the dimensions discussed in part 2 of this text. The 154 items 
were analyzed - s^arately - by three researchers. In reviewing the items> the researchers attempted to 
classify each item on each one of the 10 dimwisions. An inter-rater reliability was obtained > .8^. If 
there was discussion in relation to the categorization of a specific item along a dimension, discussion 
resulted in a consensus on the final evaluation of the item. GroiQ)iag the items along the knowledge profile 
dim^ions helped to calculate specific subscores. To ease conq>arison of mean total subscores, the 
individual subscores have been calculated as % -scores. 



4 Discussion of the research results 
4.1 General Results 

Table 2 on the next page gives an overview of the mean scores and subscores for the entire test and the 
differ^t regrouping of items along the 10 dimension^. The name of each dimension is giv^ in the first 
column. Next the names of the differ^t parameters along the dimensions are recited, with - in the third 
column - the number of items that have be^ identified as exponents of this parameter. Another strildng 
fact is the large difference in the number of items that help to calculate the parameter-subscores. Some 
Niiaa-values are even problematic. The "Reason" parameter along the "Node Relation" dimension is e.g. 
rq>resented by only two items, making this parameter less useful and weakening the validity of this 
dimension. This has to be taken into accoimt when interpreting the analysis results. 

The mean % score of the total research sample for each specific parameter is reported in the fourth column. 
The mean % scores for the different parameters in relation to each dimension show sometimes striking 
differences. The subtopic dimension presents for instance mean % scores varying from 18.3 % to 44.37 %, 
This suggests - a first level - that some dim^isions/parameters can help to indicate mastery or non-mastery 
of compon^ts of the prior knowledge state. Next to the differences in mean scores, especially the large a- 
values draw our attention. These large values are the result of the fact that the test measures " the prior 
knowledge state" of students with a wide variety of prior experi^ces in relation to the topics assessed by 
the test. In the sixth and seventh column, the mean % scores of our specific research populations are 
reported. First the scores of the 25% group are reported, next th^ of the 50 % group. The same striking 
differences in mean % scores and a-values can be observed. 



* The normal acoring procedure for this test implies that students obtsin + 1 when their answer is correct; obtain 0 when they answer with 
? and obtain -1 when their anrwer is wrong. In the recoding process, scoring for wrong answef:9 was changed into a zero-score. 

pS 0.5 ♦♦=pS .01 

* The concept •significant" in this report refers to 'statistically sigtiificam" «*«ult8. 
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4«2 Profiles of stud^ts with low (LE) and high (HE) expertise : a first analysis 



MMin % Scor«« 




Prof I les 

I LOW SHI HfQh 



Figure 2 : Profiles subpopulations 

As expected there are striking differ^ces between the means of both subpopulations. T-tests or analysis of 
variance* confirms these significant difrer^ces (F2555 = 796.4 ppjsss — -^^J ^sa% — 970.7 ppso% — -000)^ 

Wh^ discussing the profiles in relition to the differ^t dimensions, two approaches will be adopted. First, 
the profiles of both subpopulations (50% -50% and 25% -25%) will be compared. Next we will enter into 
the profiles and analyze particularities of the overall profile structure (indep^dent of the subpopulations) or 
striking differences in-between the parameters. The latter analysis will be complemented in die next part of 
this text wh^ executing a flatness test based on a multivariate analysis of variance (profile analysis). 

4.2.1 Economics subdomains dimension 

The data in fig. 3 and 4 reveal clear differ^ces in the mean % scores of students with a low (LE) and a 
high prior knowledge state (HE) for the different economics-subdomains. As expected, the mean % scores 
of HE are always higher for all subdomains. If we neglect the intenel^tions betwe^ the different 
economics-subdomains, and test the significance of the differ^ces between the mean-scores^ we find that 
these differ^ces are always significant (p? < .000)', 



*For most parameters, the variances of the 25%-25% group are homogeneous and an analysis of variance can be executed. But for 
the 50%-50% groups the variances are not homogeneous. As a result only t-tests will be calculated to compare the mean % scores of 
the subpopulations in relation to this subvision in the population. Ln this pait of the text we will. At the end of this pait (4.2), we will 
sunMnarizc thv nalysis findings. The reader will find whether t-^sts or variance analysis have been applied. 

* The concept •significant* in the further part of this text refers to "statistical significance*. 

^ In part 5.3 of this text, we will take the intercorrelation between the subdomains into account when executing a parallelism test» 
based on a multivariate analysis of variance (profile atialysis). 

* For an overview of the F- and prvalues, we refer to table 3. 
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Figure 3 : Economics Subdomains knowledge profiles (50% -50%) 
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Figure 4 : Economics Subdort \ins knowledge profiles (25%-25%) 

There is a remaricable difference in the profile-con^>arison of the 50% -50% and 25 %-25% groups. In the 
latter, the mean % score of the HE-group for the subdomain micro-economics does not differ to a large 
extent from the grand mean. Also for the subsequent subdomains (Public finances, International economic 
affairs and Behavioiirial and social sciences) this difference is not that large. This might indicate that for 
these subdomains, a ceiling effect is noticed. 

The differ^ce between the mean % scores of the LB group is analogous in relation to all subdomains. 
4.2.2 Curriculum accent dimension 



The difference in mean % scores of HE and LE students is consistently significant for both curriculum 
accent parameters. When con^aring the profiles, it is noticeable that the mastery of general economics of 
LE students is very much lower than the grand mean. 
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Figure 5 : Curriculum accent knowledge profile (50%-50%) 

The curriculum accent profiles show that "general economics*' mastery is higher than "quantitative 
economics' mastery for all subpopulations. This can indicate that the prior knowledge state is of more 
importance v/b&a solving quantitative economics problems. But» this can also be explained by linking these 
results to earlier research findings. When analyzing e.g. the "components" of the prior knowledge state in 
the field of economics^ it was found that a mathematics related prior knowledge state helped to explain to a 
significant and relevant extent the variance in economics prior knowledge state test scores (Dochy> Valcke 
& Wagemans, 1991 & Wagemans, Valcke & Dochy, 1991). 
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Figure 6 : Curriculum accent knowledge profile (25%-25%) 
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4*2.3 Curriculum level dimension 

Mastery of both curriculum levels is always higher for HE students than for LE students. These differences 
are significant. 
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Figure 7 : Curriculum knowledge profile (50%-50%) 



As expected, the mean % score for the level-2 parameter is lower for both sub-populations. It is normal 
that prior knowledge of these advanced level questions is restricted. An additional explanation questions the 
quality of the level-2 items. It is possible that these are of a much higher order. 



Mean % Scores 




Level 1 



Level 2 



Course Level 




Figure 8 : Curriculum knowledge profile {2S%-2S%) 
4.2.4 Node relation dimension 

There is a consist^t significant diffitence in the mastery of all node relation parameters between LE and 
HE students. Although significant, these differences are less large in the 25%-25% population. This is 
especially true for the parameters initiate, consequence, manner and property. 
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Figure 9 : Node relation knowledge profile (50%-50%) 

In both figures^ it is remarkable that the mean % scores for the different parameters do differ to a large 
extent. Items» classified as "reason" or '*manner** are more easily solved than the other node relation 
parameters. The flatness test will indicate whethei these in-profile differences are significant. 
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Figure 10 : Node relation knowledge profile (2S%-2S%) 
4.2.5 Behaviourial dimension 

The differences between the mastery of LE and HE students are significac'i at all levels. 
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Figure 21 : Behaviourial knowledge profile (50%-50%) 



22 



Low and high leveb of the Prior Knowledge SUte 



At the theoretical level, ii was indicated that this dim^ision is hierarchical. This might imply that items, 
classified according to the "apply" parameter are more difficult than e.g. items classified as "knowing*. 
This is confirmed, when looking at the two figures. Mastery of higher level items is always lower than 
lower !eve! items. We can also notice a ceiling effect in relation to difficult items : the mean % scores of 
HE student is not this different from the grand mean in the 2S%-2S% group. 
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Figure 12 : Behaviourial knowledge profile (25%-25%) 
4.2.6 Content dimension 

The differences between the mastery of LE and HE students are significant in connection to all parameters. 
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Figure 13 : Content knowledge profile (50%-50%) 
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Figure 14 : Content knowledge profile (25%~25%) 

Parameters, along the content dimension are considered as being hierarchical. Both pictures show some 
remaricable particularities in this perspective : the mastery of items linked to the "fact" parameter is 
consistently lower than the mastery of items linked to the "concq>ts, relations, structures and methods" 
parameters. We can explain this difference in mastery by referring to the difficulties in retaining and 
remembering isolated factual knowledge. Whereas, concepts are more easily recalled since at this level 
facts are embedded in a structure, scheme. But, since these structures or schemes become more complex 
when they are integrated into relations or structures, mastery of them decreases. Both figures confirm this 
t^dency. 

The mastery of items, classified according to the "methods" parameter, follows the overall trend described 
above. 

4.2.7 Epistemological dimension 

The knowledge profile of HE students is consistently and significantly higher than the profile of LE 
studeLts. 
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Figure 15 : Epistemological knowledge profile (50%-50%) 



ERIC 



24 



Low tnd high levels of the Prior Knowledge Suie 



p.20 



(25%-25%) 



Mean % Scores 




<now.i(Jen!. Know.concept. Eplst. anal. Loolcal finol. impUana 

Epistemologlcal Levels 



Low axportlae High exportlao Grand motn 



Figure 16 : Epistemological knowledge profile (25%-25%) 

This dim^ision is a combination of the behaviourial and content hierarchical dimension. It is expected that 
analogous profiles will be found. Fig. 15 and 16 confirm our expectations. We can even slate that the 
pattern in the particular expected profiles is reinforced. When looking at figure 16, we perceive that the 
mean % score of the HE stud^ts does hardly differ from the grand mean when looking sit the parameters 
**Logical analysis and Implementional analysis". This can be explained as follows : the sMd^ts in our 
research sample have - yet - not been in the position to act as economists, thus dealing with real-life 
problems which have to be solved by applying higher order economics knowledge. Since experi^ce is 
needed to consolidate the mastery of this type of economics knowledge, we cannot expect high prior 
knowledge state scores. 

4.2.8 Number of propositions dimension 

Items with a high number of propositions are more complex than items with a low number of propositions. 
It is expected that HE students perform better at all proposition-levels than LE students. This is confirmed 
by the profiles in figures 17 and 18. The differences in performance are significant. If we look at the 
25% -25% group, we see that the mean % score of the HE group does hardly differ from the grand mean. 



(50%-50%) 

Mean % Scored 

40^ 

30 
20 
10 

0 J 
^fOpoai Prop032 Propooc 

Number of Propositions In question 
— ^ Low exportiee — High expertlao Grand mean ■ 




Figure 17 : Number of propositions knowledge profile (50%-50%) 



BEST COPY AVAILABLE 



o 25 

ERIC 



Low tod high levels of the Prior Knowledge State 



p.21 



(25%-25%) 



Mean % Scores 




^'roposl Prop032 Propose 

Number of Propositions In question 

Low expertise — ^ High ©xpertlao QrorxJ rrean M 



Figure 28 : Number of propositions knowledge profile {25%-25%) 

^ ,ure 17 and 18 also illustrate that the mastery of items ordered along the subsequent parameters gradually 
declines. 

4.2.9 Information level dimension 

Consistent and significant differences are found between L£ and HE students. 
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Figure 19 : Information level profile (50%-50%) 
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Figure 20 : Information level profile (25%-25%) 

Referring to the theoretical discussion in relation to this dimension, we expect that items without a stem are 
more difficult than items with a stem. Figure 19 and 20 confirm our expectations. The pattern found, is 
also consistent with the findings in figure 17 and 18 in connection to the number of propositions dimension. 

4.2.10 Representation level dimension 

Consistent and significant differences are found between LE and HE stud^ts in relation to all parameters. 
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Figure 21 : Rqjrcsentation level knowledge profile (50%-50%) 
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Figure 22 : Representation level knowledge profile (25%-25%) 



The profiles suggest that all students are more able to solve questions based on graphical information. 
Addition of schematic and symbolic information to text-based items, seems not to enhance performance of 
students. The profiles even suggest that such items are even more difficult than purely text-based items. 
In addition, we perceive in figure 22 that the mean % score of the HE students (22.6%) for items classified 
as "textual-symbolic" is hardly different from the grand mean (20.5%). 

4.2.11 Intermediate conclusions 

All dimensions help to differentiate betv^een LE and HE stud^ts. All differences between both student 
populations are significant, whether we consider the 25%-25% or the 50% -50% group. The reader might 
conclude that the dim^ions have not been helpful to detect very specific contrasts between LE and HE 
students. But, the specification of the 25%-25% groiq) has proven to be useful since more distinctive 
differences between low and high performers could be detected than con^aring the 50%-50% group. The 
fact we tried to avoid deviation of the distribution of the mean % scores by excluding extreme scoring 
students from the research sample, has been successfiil to a certain ext^t. 

Expectations about the knowledge profiles - based on our theoretical considerations - have been largely 
confirmed (e.g. hierarchical nature, subsequent difficulty levels, etc.). 
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TabU 3 : 

Comparison of mean % scorea of LE and HE students in the 75%'Q5% group 
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TabU 4 : 

Comparison of mean % scores of LE and HE students in the 50%-50% group 



DimensiOQ/parameter 
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4.3 'PA>iile analysis 

A univariate analysis of variance (as used in part 4.2 of this text) does not take into account the 
intercorrelations betwe^ the differ^t parameters along the profile dim^isions. These interconrelations are 
important (although not making the specific variables redundant) and can be explained at the theoretical 
level as illuminated elsewhere (cf. Dochy & Valcke» 1991a). A multivariate analysis of variance is needed 
to refine our analysis and to look for more conclusive information about the differences in the prior 
knowledge state betwe^ LE and HE students. A multivariate analysis can take these interconrelations into 
account* Profile analysis is an ext^ion of multivariate analysis and is especially ^appropriate and helpful to 
evaluate the parameter structure in relation to each profile dimension when conqiaring subpopulations. 
Several tests are available in profile analysis. Of principal interest - for oar purposes - is the "parallelism" 
test which help to answer the question whether the profiles of two subpopulations are parallel or no>'. If 
certain dimensions are helpful to detect non-parallel profiles, it might be interesting to know what parameter 
along the diinension does contribute most to these significant differ^ces. Therefore, for non-parallel 
profiles, the analysis will be ext^ded with a discriminant analysis (also called, the "level" test). At the 
theoretical level - as suggested in part 4.2 of this text • also the "flatn^" test might be relevant, since this 
test controls the similarity of responses for the different parameters along a dim^ion, indep^dent of 
groups or subgroups. An answer to this question helps to siqsport the validity of the different dimensions 
since the results indicate whether or not the dimensions/parameters are helpfU to specify differences in the J., 
mastery of differ^t components of the prior knowledge state. 

A profile analysis will be performed on the con^)lex of parameters in relation to each dimension. The 
grouping variable is defined by LE and HE. SPSS-PC^ MANOVA was used for our profile analysis. 

4.3. 1 Control of underlying assumptions 

Profile analysis implies that specific assumptions about the quality of the research data are met (no missing 
data, comparable san^le sizes, (multivariate) normal distributions, no outliers, homogeneity of variance- 
covariance, multicollinearity). 

Data screening revealed no missing data. 

Sample sizes are nearly equal for all subpopulations (Nlejs* - J ^iie2s% = 153: ^hES(i% = 



I^HESDs ~ ^01) ^ special difficulties are expected. Moreover, only one indq)endent 
variable is used. 

The evaluation of the homogeneity of variance-covariance matrices is based on the Cochians C and 
the Bartlett-Box F test. 

To evaluate assun:q>tions about multivariate normality, boxplots of the mean submeasures for each 
dim^ioii have been screened. 

Multicollinearity is tested with the Bartlett test of sphericity. 

The research data for the 25%-25% and 50%-50% group have been screened separately. 

In evaluating multivariate normality of the distribution of the mean %-scores, it is to be mentioned that the 
a-values are very high, indicating a wide dispersion of the scores (cf. table 2). This is to be expected, since 
the test measures " the prior knowledge state". Figure 13 presets e.g. a box-plot of the scores of ES and 
LS for the subtopic "marketing" on the "Economics subdomain" dim^ion. llie wide dispersion of the 
scores is obvious. The id^tifies the median and the box contains the middle 50% of the values. 
The lines emanating from the box extend to the smallest and largest observations in the subgroups that are 
less than one interquartile range from the end of the box. Points outside this range are marked with O 
(outliers) or even E (Extremes) if more than l.S interquartile distances away from the box. 

Analysis of the box-plots for each variable in relation to each profile dimension reveals that there are 
outliers and extremes. This is especially true for the 50% -50% groups. As argumented earlier in this text, 
a new subdivision of the large student sample has been effected in order to diminish the impact of these 
extreme scoring students in the research sample. The box-plots of the 25% -25% groups depict as a 
consequence less outlying and extreme values. 
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Table S and 6 on the next page summarize the data in relation to the evaluation of the homogeneity of 
variance-covariance matrices and the multicoUinearity test. 

As can be concluded from table 6» r^res^ting the data for the SO%-SO% groups, the Cochrans C and 
Bartlett-Box F values are significant in relation to all dimensions and nearly all parameters. This makes a 
straightforward interpretation of the multivariate analysis of variance results hardly possible. As a 
consequ^ce we prefer not to involve the 50% -50% groups subdivision in the further analyses. 
When analyzing the data in relation to the 25%-25% groiq)s» only minor violations against the homogeneity 
of variance-covariance are observed (subdomains» level and node relation dimension). But since the 
sample sizes are sufficiently large and comparable, no problems are expected. 

The Bartlett test of sphericity is significant in all cases, ev^ within tb .) 2S%-25% groups, which means that 
the variables are highly intercorrelated. Although the p-values are very small, the SPSS-MANOVA-PC^ 
procedure protects against instability caused by multicollinearity by excluding variables from the analysis 
with too low tolerance levels^ The fact, the MANOVA-procedure was never halted during execution 
indicates that multicoUinearity did not cause problems. 

In general we can summarize that ivssumptions are met in order to execute a profile analysis on the research 
data available if we restrict our analysis to the 25%-25% groups. 



* Tolerance level ™ 1 - SMC (squared multiple correlation of each variable). 
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Table 5 : 

Analysis data in relation to multicoUinearity 
and homogeneity of varianco-covariance matrices (25%-25%) 



ERLC 





Homogeneity of Variance 


MulticoUinean^y 








Bartlett test of sphericity 




CocbranaC 


Bartlett-Box F 






.52264 (p=. 582) 


.30172 (p=. 583) 




Finance 


.50310 (p=. 940) 


.00564 (p=. 940) 




Organ 


.53886 (p= .344) 


.88855 (p= .346) 




Mari^et 


.53533 (p=.390) 


.73621 (p=.391) 




Macro 


.57166 (p=. 080) 


3.03883 (p«.081) 


275.58412 (p=. 000) 


Micro 








Public 


51 133 fn= 19X\ 


.VI/ JJJ \p— ./<w^ 




Interoat 


52511 ro=s 542^^ 


')70d0 (nez SdVi 




Behav 


608QQ (n= (Vrn 


7 t tfiOA ^n— n^fl^ 

/.I loVW »\J\JO) 




Level 1 


.63825 (p«.O01) 


11.61489(p=.001) 


296.25458 (p=. 000) 


UveI2 


J9317(p«.022) 


5.17010^=.023) 




Quant 


.566q9-^p=.107) 


2.58189 (p=. 108) 


22.47357 (p~. 000) 


General 


.56172 (p*;i32) 


2.24948 (p==. 134) 




Reason 


.64194 (p = . 000) 


12.26879 (p=. 000) 




Initiate 


J7858(p=.055) 


3.66137 (p=.056) 




Conseq 


.58794 (p = . 031) 


4.59869 (p=. 032) 


60.03420 (p=. 000) 


Manner 


.53088 (p = . 453) 


.56060 (p=. 454) 




Property 


.50125 (p=.976) 


.00092 (p=. 976) 




Know 


.53312 (p=. 421) 


.64479 (p=,422) 




Insight 


.55385 (p=.190) 


1.70983 (p=.191) 


55,60628 (p=. 000) 


Apply 


.59257 (p = . 023) 


5.10368 (p=.024) 




Factual 


.55243 (p=. 202) 


1.62028 (p=. 203) 




Concept 


.54704 (p=. 252) 


1.30341 (p=.254) 




Relat 


.51130(p>=.784) 


.07511 (p=.784) 


96,36344 (p=. 000) 


Stmct 


.57648 (p=» .062) 


3.46616 (p<= .063) 




Methods 


.56499 (p = . 113) 


2.49574 (p=. 114) 




Kident 


.50769 (p«. 852) 


.03472 (p= .852) 




Kconcept 


.54947 (p = . 228) 


1.44159 (p=. 230) 




l^isto 


.59939 (p = .015) 


5.89752 (p=. 015) 


88.89663 (p = , 000) 


Logical 


.57297 (p = . 075) 


3.15242 P=.076) 




Implem 


.59131 (p=.025) 


4.96308 (p». 026) 




Proposl 


.59374 (p=. 022) 


5.23483 (p=. 022) 




Prcpo82 


.58138 (p = .047) 


3.93047 (p=. 048) 


96.35997 (p=. 000) 


ProposS 


.57109 (p=.083) 


2.99083 (p=. 084) 




Witha 


.60097 (p«. 013) 


6.08923 (p<=:. 014) 


50,61913 (p<=, 000) 


Withouta 


.57527 (p=. 066) 


3.35610 (p=.067) 




Text 


.52252 (p=. 584) 


.29801 (p«.585) 




Concret 


.56315 (p=. 124) 


2.35568 (p«. 125) 


76,13433 (p=.000) 


Scheme 


.56284 (ip=. 125) 


2.33262 (p=. 127) 




Symbol 


.54343 (p =.2'^ I) 


1.11025 (p=.292) 




33 



Low and hi^ levels of the Prior Knowledge State 



p.29 



Table 6 : 

Analysis data in relation to multicoUinearity 
and homogeneity of varianco-covariance matrices (50%'50%) 





. Homogeneity of Variance 


MulticoUinearity 
Bartlctt test of sphericity 




CochransC 


Bartlett-Box P 


Report 

Finance 

Oigan 

Ma^et 

Macro 

Micro 

Public 

Intemat 

Behav 


.55951 (p=.017) 
.58297 (p=. 004) 
.54908 (p = . 091) 
.50558 (p=. 848) 
.61940 (p = . 000) 
.70183 (p=.O0O) 
.53973 (p=. 172) 
.61233 (p = . 000) 
.73090 (p = . 000) 


5.72573 (p=. 017) 
8.19001 (p=. 004) 
2.84202(p=.092) 
.03666 (p=. 848) 
17.20529 (p=. 000) 
52.01233 (p=. 000) 
1.86011 (p=. 173) 
15.17855 (p=. 000) 
70.08597 (p=. 000) 


960.22167 (p=. 000) 


Levell 
Level2 


.56066ft) 037) 
.73824(1)= .000) 


4.35169(p=.037) 
75.23063(p = .00O) 


29.91204 (p=. 000) 


Quant 
General 


.63419 (p=». 000) 
.66160 ft)=. 000) 


21.89541 (p=. 000) 
. 32.29598(p = .134) 


94.61048 ft) = .000) 


Reason 

Initiate 

Conseq 

Manner 

Property 


.66324 (p = . 000) 
.69352 (p=. 000) 
.65119 (p = . 000) 
.56572 (p=. 023) 
.65112 (p=. 000) 


32.99057 (p=. 000) 
47.46595 (p=. 000) 
28.07583 (p=. 000) 
5.11414 (p=. 024) 
28.04990(p=.000) 


504.97102 (p = .000) 


Know 

Insist 

Apply 


.60879 (p = . 000) 
.64560 (p=« .000) 
.69796 (p«. 000) 


14.21446 (p=. 000) 
25.94753 (p=. 000) 
49.86202 (p=. 000) 


384.32928 (p=. 000) 


Factual 

Concept 

Relat 

Stnict 

Methods 


.67159 (p = . 000) 
.52773 (p=. 341) 
.56723 (p«. 020) 
.69566 (p = . 000) 
.68988 (p = . 000) 


36.66917 (ip = . 000) 
.90505 (p=. 342) 
5.35307 (p=. 021) 
48.61243 (p=. 000) 
45.55333 (p=. 000) 


646.84117(p=.000) 


Kident 

Kconccpt 

Episto 

Logical 

Implem 


.57685 (p=. 008) 
.53637 (p=. 211) 
.66301 (p = . 000) 
.64039 (p = . 000) 
.70179 (p = . 000) 


7.01218 (p=.008) 
1.55762 (p=. 212) 

32.89280 (p=. 000) 
24.05 157<p=. 000) 

51 . 98872 (p = . 000) 


525.73181 (p = . 000) 


Proposl 
Propo82 
Ptopos3 


.62510 (p— .000) 
.58310 (p=. 004) 
.71850 (p«. 000) 


18.94130 (p= .000) 
8.21516 (p=. 004) 
61.94216 (p=.000) 


447.41569 (p=. 000) 


Withs 
Withouto 


,67048 Op = .000) 
.61901 (p=.000) 


36.16804 (p». 000) 
17.09039 (p=. 000) 


262.10691 (p=. 000) 


Text 
Concret 
Scheme 
Symbol 


.62132 (p=. 000) 
.56635 (p=. 022) 
.59133 (p = . 002) 
.71075 0?=. 000) 


17.78058 ft)= .000) 
5.21240 (p=.023) 
9.95045 (p=. 002) 

57.18240 (p=. 000) 


365.72403 (p=. 000) 
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4.3.2 Profile Analysis Results : Parallelism Test 

Table 7 presets an overview of the profile analysis results in connection to the parallelism test. This helps 
us to answer the question whether the two differ^t student groups (L£ and HE) have parallel or non- 
parallel profiles. This is commonly known as the test of parallelism and is the primary question addressed 
by profile analysis^ In relation to each profile dimension, Wilk's Lambda (X) was calculated and p-levels 
determined. In the results table, Wilk's \ is not reported in relation to three dim^isions (marked with *^). 
This is because these dimensions only contain two variables; in these cases a test of significance for 
Hoteling's T2, using the unique sums of squares, was calculated checking the interaction of the independent 
variable (LE and HE) and the two dependent variables on the specific dimensions. 

Table 7: 



Results of the parallelism test in profile analysis 



DIMENSION 


^^Iks XorF 


Pxor Pp 


Bconomics 
subdomaina 


.96459 


.232 


Curriculum level* 


20.94 


.000 


Curriculum accent* 


536.17 


.003 


Node relation 


.96662 


.041 


Behaviourial level 


.93784 


.659 


Content level 


.97667 


.140 


Epistemologtcal 


.24876 


.000 


Number of 
propositions 


.79462 


.000 


Information level* 


53.55* 


.000 


Representation level 


.95832 


.006 



The data in table 7 help to detect specific significant differ^ces in knowledge profiles of LE and HE 
stud^ts. Significant differences are rq>orted in relation to the following ditn^ions : course level, 
curriculum level, behaviourial level, number of propositions, information level and rqiresentation level. 
Therefore, the hypotheses stated in part 3.1 of this text can be confirmed to a very large extent. 
Since the results indicate non-parallel profiles, a discrimmant analysis and the calculation of structure 
coefficients are relevant. 

4.3.3 Discriminant Analysis : Structure Coeffici^ts 

As said in the introduction to profile analysis, a further analysis of non*parallel profiles might be interesting 
in order to know what specific parameter(s) along the dim^ion does or do contribute most to these 
difTer^ces in profiles. Table 8 presents the results of a discriminant analysis. In the second and third 
column a new value for Wilk's \ is reported and its significance level^. Moreover, Wilk's X can - in this 
context - be inteipreted as a measure indicating the proportion of variability not explained by the group 
differences. In the fourth column, we derive from X ((1 - X) 100) the proportion of variability that is 
explained by the group differences based on the independent variable (LE and HE). 



^ When L*<<ing profile analysis as a substitute for univ&ri&te repeated measures ANOVA, the parallelism test is the test of interaction. 
' Of course this futther analysis is limited to the dimensions resulting in non-parallel profiles. 



ERIC 35 



I^w and high levels of the Prior Knowledge State 



p.31 



TabU 8 : 
Results of the discriminant analysis 



DIMENmON 


VfSks X 


Pk 


1 1" 1 ■ 


Curriculum 
level* 


.26612 


.000 


73 % 


Curriculum 
accent* 


.26933 


.000 


73 % 


Epiftemological 


.24876 


.000 


75 % 


Number of 
pn^K>sitions 


.26331 


.000 


73 % 


Information 
level* 


.26167 


.000 


73 % 


Representation 
level 


.27316 


.000 


72 % 



Since this extension of our profile analysis indicates significant X, the analysis can be continued by 
calculating structure coefficients* to determine the discriminatory power of the separate values for each 
dimension parameter. The results of this further analysis are rq>orted in table 9. Only the most relevant 
structure coefficients in relation to each dimension are reported. 

Table 9 : 

Overview of most relevant structure coef&cients 



Dimension 


Parameter 


Stn^tore coeff. 


Curriculum level 


Levell 


.371 


Curriculum accent 


General 


.917 


Epistemological 


Epiat. an. 


-,524 


Number of propositions 


< 5 prop. 


-.472 


Information level 


With stem 


.701 


Representation level 


Textual 


-.679 



The results in table 9 could be predicted since the structure coefficients reflect the large F-values we already 
obtained when reporting the results of the tmivariate analysis of variance in mean % scores in part x of this 
text. 

4.3.4 Profile Analysis Results : Flatness Test 

Is the mastery of the prior knowledge state as defined by the parameters along a dimension different, 
independent of the groups (a within-subjects main effect) ? In other words, do students master the prior 
knowledge state in a similar way as defined by the differ^t parameters along a dimension 7 This question 
is especially relevant for parallel profiles, since in non-parallel profiles at least one parameter is not flat; 
nevertheless also the results in relation to non-parallel profiles are reported. 



* Since the lubvalueo on each profile dimension are highly Intercorrelated, we cannot use raw or standardized discriminant function 
coefHcientfi. The highly correlated variables "share* the discriminants weights. It is safer to base our intcrpretstton on the structure 
coefficients which are less likely to be influenced by these intercorrelattons. 
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Table 10 : 

Results of the flatness test in profile analysis 



DIMENSION 


mksXorF 




EcoDomict 
fubdomatns 


.33278 


.000 


Curriculum 
level* 


2415.11* 


.000 


Curriculum 
accent* 


26.46* 


.000 


Node relation 


.50081 


.000 


Bchaviourial 
level 


.59964 


.000 


Content level 


.42087 


.000 


^istemological 


.49229 


.000 


Number of 
propositions 


.05396 


.000 


Infonnation 
level* 


2760.67 


.000 


Representation 
level 


.46463 


.000 



If the flatness test is non-significant> then the profiles are not helpfiil to clarify or detect differences in the 
mastery of different conq)onents of the prior knowledge state. The results of the flatness test are therefore 
also of relevance to determine the validity of the knowledge profile dimensions. 
The results of the flatness test are found in table 10. For each dimension Wilk's X has been calculated, 
with the excq)tion of the three dimension where only two parameters are available along the dimension; 
there the F-value is reported (mariced with *)♦ All dimensions result in non-flat knowledge profiles. This 
implies that all dimensions are helpfid to identify a specific structure in the mastery of the prior knowledge 
state. Following this structure, the mastery of certain components* of the prior knowledge state is better 
than for other components. 



5 Conclusions 



In this text, we attempted to analyze the prior knowledge state of two specific Open university 
subpopulations, studying a multi-functional course. Earlier research helped to confirm that the prior 
knowledge state differences do exist between these economics and law students. In the preset study, the 
overall economics score was not significantly different between both student groups. But in this research, 
special attention was paid to a further elaboration of this general economics-score by grouping items along a 
variety of profile dimensions. Although a profile analysis could not help to reveal specific significant 
differences between the two subpopulations, the present study is of high importance since we succeeded in 
defining and operationalising a new more promising ^proach towards the analysis of the prior knowledge 
state. It is foreseen that in situations where there are significant differences betwera the prior knowledge 
state of specific subpopulations, the profile dimension might be helpful to detect and dissect the strengths 
and weaknesses of die studrats involved. This might be a promising starting point for differentiated 
diagnostic and guidance approaches. 



* The concept 'components* refers to this subpart of expertise that can be isolated Ln connection to a specific parameter along a 
knowledge profile dimension. 
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